add trustless IPFS gateway, use patched rclone, remove bitswap#646
Open
add trustless IPFS gateway, use patched rclone, remove bitswap#646
Conversation
remove the dead bitswap protocol and add an /ipfs/ endpoint implementing the trustless gateway spec via boxo/gateway. StorageBlockStore pools rclone handlers per storage and holds open a streaming file reader to serve sequential block reads efficiently -- DAG nodes return from the DB without storage I/O, and leaf blocks from the same file share a single held-open rclone stream. supports dag-scope=all/entity/block, path resolution, entity-bytes, format=car and format=raw. stale source files detected at read time are mapped to HTTP 409 via boxo's error status mechanism.
parkan
force-pushed
the
feat/trustless-gateway
branch
from
fe93173 to
adfce2b
Compare
parkan
changed the title
replace github.com/rclone/rclone with parkan/rclone@singularity-v1.68.0 which adds sync.Map + singleflight caching for /metadata/ API calls in the internetarchive backend. reduces per-item metadata hits from N (one per file) to 1. temporary measure until metadata caching is upstreamed to rclone.
- remove uptobox backend (dropped upstream) - fix fs.Duration type change in config overrides - handle []string option defaults in ToCLIFlag type switch
parkan
force-pushed
the
feat/trustless-gateway
branch
2 times, most recently
from
996b763 to
51b8666
Compare
parkan
force-pushed
the
feat/trustless-gateway
branch
from
51b8666 to
a8d4dd4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
remove the dead bitswap protocol and add an /ipfs/ endpoint implementing the trustless gateway spec via boxo/gateway
StorageBlockStore pools rclone handlers per storage and holds open a streaming file reader to serve sequential block reads efficiently -- DAG nodes return from the DB without storage I/O, and leaf blocks from the same file share a single held-open rclone stream
supports dag-scope=all/entity/block, path resolution, entity-bytes, format=car and format=raw. stale source files detected at read time are mapped to HTTP 409 via boxo's error status mechanism